Hadamard product (matrices)

In mathematics, the Hadamard product is a binary operation that takes two matrices of the same dimensions, and produces another matrix where each element ij is the product of elements ij of the original two matrices. It should not be confused with the more common matrix product.

The Hadamard product is associative and distributive, and unlike the matrix product it is also commutative. The Hadamard product is named after French mathematician Jacques Hadamard), and is also known as the entrywise product and the Schur product.^[1]

1 Definition
2 Properties
3 Schur product theorem
4 In programming languages
5 Applications
6 See also
7 References
8 External links

Definition

Formally, for two matrices of the same dimensions

$A, B \in {\mathbb R}^{m \times n}$

the Hadamard product A ○ B is a matrix of the same dimensions

$A \circ B \in {\mathbb R}^{m \times n},$

with elements given by

$(A \circ B)_{i,j} = (A)_{i,j} \cdot (B)_{i,j}.$

Properties

The Hadamard product is commutative, associative and distributive over addition. That is,

$A \circ B = B \circ A,$

$A \circ (B \circ C) = (A \circ B) \circ C,$

$A \circ (B %2B C) = A \circ B %2B A \circ C.$

The identity matrix under Hadamard multiplication of two m-by-n matrices is m-by-n matrix where all elements are equal to 1. This is different from the identity matrix under regular matrix multiplication, where only the elements of the main diagonal are equal to 1. Furthermore, a matrix has an inverse under Hadamard multiplication if and only if none of the elements are equal to zero.^[2]

For vectors $x$ and $y$ , and corresponding diagonal matrices $D_x$ and $D_y$ with these vectors as their leading diagonals, the following identity holds:^[3]

$x^*(A \circ B)y = \mathrm{tr}(D_x^* A D_y B^T)$ ,

where $x^*$ denotes the conjugate transpose of $x$ . In particular, using vectors of ones, this shows that the sum of all elements in the Hadamard product is the trace of $AB^T$ . A related result for square $A$ and $B$ , is that the row-sums of their Hadamard product are the diagonal elements of $AB^T:$ ^[4]

$\sum_j (A \circ B)_{i,j} = (AB^T)_{i,i}.$

The Hadamard product is a principal submatrix of the Kronecker product.

Schur product theorem

The Hadamard product of two positive-semidefinite matrices is positive-semidefinite.^[5] This is known as the Schur product theorem,^[2] after German mathematician Issai Schur. For positive-semidefinite matrices A and B, it is also known that

$\det(A \circ B) \ge \det(A) \det(B). \,$ ^[5]

In programming languages

Hadamard multiplication is built into certain programming languages under various names. In MATLAB, it is known as array multiplication, with the symbol .*.^[6]

Applications

The Hadamard product appears in lossy compression algorithms such as JPEG.

References

^ (Horn & Johnson 1985, Ch. 5)
^ ^a ^b Million, Elizabeth. "The Hadamard Product". http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf. Retrieved 2 January 2012.
^ (Horn & Johnson 1991)
^ (Styan 1973)
^ ^a ^b (Styan 1973)
^ "Arithmetic Operators + - * / \ ^ ' -". MATLAB documentation. MathWorks. http://www.mathworks.se/help/techdoc/ref/arithmeticoperators.html. Retrieved 2 January 2012.

External links

The Hadamard Product